Papers with dense encoders
Dial-MAE: ConTextual Masked Auto-Encoder for Retrieval-based Dialogue Systems (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing studies on dialogue response selection focus on post-training and fine-tuning for cross-encoders. |
| Approach: | They propose a post-training technique tailored for dense encoders in dialogue response selection . they propose 'Dialogue Contextual Masking Auto-Encoder' which compresses dialogue semantics into dense vectors . |
| Outcome: | The proposed technique achieves state-of-the-art on two commonly evaluated benchmarks. |
Weakly Supervised Pre-Training for Multi-Hop Retriever (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for weakly supervised multi-hop pretraining require costly human annotation. |
| Approach: | They propose a method for weakly supervised multi-hop retriever pretraining without human efforts by generating vector representations of complex questions and subquestion as weak supervision for pre-training. |
| Outcome: | The proposed method is effective and robust on limited data and computational resources. |
RegTrack: A Fine-Grained Benchmark for Multi-Class Legal Change Detection (2026.acl-srw)
Copied to clipboard
| Challenge: | Existing models lack finegrained classification schemes to determine whether small changes impact legal obligations or merely update formatting. |
| Approach: | They propose a benchmark for change detection in EU regulations that uses 4,772 manually annotated pairs of structurally distinct provisions mapped to a six-class taxonomy of legal change types. |
| Outcome: | The proposed framework combines lexical algorithms, dense encoders, and Large Language Models (LLMs) as baselines. |
Condenser: a Pre-training Architecture for Dense Retrieval (2021.emnlp-main)
Copied to clipboard
| Challenge: | Prior work fine-tunes deep LMs to encode text sequences into single dense vector representations, but dense encoders require a lot of data and sophisticated techniques to train and suffer in low data situations. |
| Approach: | They propose to pre-train Transformer language models (LMs) with a novel Transformer architecture, Condenser, where LM prediction CONditions on DENSE Representation. |
| Outcome: | The proposed model improves on various text retrieval and similarity tasks by large margins over standard LMs. |
RECOR: Reasoning-focused Multi-turn Conversational Retrieval Benchmark (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing benchmarks treat multi-turn conversation and reasoning-intensive retrieval separately, yet real-world information seeking requires both. |
| Approach: | They propose a framework that transforms complex queries into fact-grounded multi-turn dialogues through multi-level validation. |
| Outcome: | The proposed framework outperforms existing systems in a number of domains and can be used to improve multi-turn conversation retrieval. |
JuriFindIT: an Italian legal retrieval dataset (2026.findings-eacl)
Copied to clipboard
| Challenge: | Statutory article retrieval (SAR) targets retrieval of legislative provisions relevant to a natural language question. |
| Approach: | They propose a pipeline that integrates dense encoders with an heterogeneous legislative graph . they propose statutory article retrieval (SAR) is the first SAR dataset for the italian legal domain . |
| Outcome: | The proposed pipeline improves over existing approaches. |